Abstract

Today, malware is among the most common and harmful kinds of cyberattacks. It affects millions of devices globally and can execute a variety of damaging operations including file encryption, confidential information extraction, system performance degradation, and so many others. The most dangerous among the malware variants is polymorphic malware. Polymorphic malware differs from simple encryption in that it can use infinite encryption techniques and modifies a portion of the decryption code with each iteration. Different malicious executions by the malware might be classified under encryption operations, depending on the type of malware. The encrypted virus usually contains an embedded transformation engine, which produces a new encryption algorithm at random. The developed algorithm integrates a new decryption key into the engine, leading to the creation of encrypted malware. Deep Learning (DL) has emerged as a promising technology for identifying such threats. Given the increasing complexity of malware variants across various network environments, DL techniques offer robust solutions for developing scalable and sophisticated models capable of detecting and classifying malware effectively, especially due to their ability to process huge volumes of data. This survey outlines the various research and efforts in application of deep learning models for polymorphic malware over encrypted networks. The survey employs systematic approach where research works and findings are presented highlighting major contributions to knowledge and key challenges and recommendations for future studies